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(54) System and method for recording and synthesizing sound and infrastructure for distributing 
recordings for remote playback 

(57) Systems and methods for recording and syn- 
thesizing sound in a resolution-independent manner 

and infrastructures for distributing resolution-independ- 1 
ent recordings for remote playback. In one embodiment, 
one system includes: a frame generator (130) that ex- 
tracts fundamental frequencies and spectral envelopes 
from the sound and creates frames therefrom and a 
frame analyzer (140) that identifies a selected one of 
common spectra structures and common formant struc- 
tures in the frames and creates a record containing the 
fundamental frequencies and the selected one. One in- 
frastructure includes: a radio station having a recording 
database associated therewith, a plurality of recordings 
contained within the recording database, each of the 
plurality of recordings including the selected one, a re- 
quest receiver, coupled to the recording database, that 
receives remote requests for ones of the plurality of re- 
cordings and a transmitter, coupled to the recording da- 
tabase, that transmits the ones of the plurality of record- 
ings in response to the requests. 
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Description 

Technical Field Of The Invention 

[0001] The present invention is directed, in general, 5 
to sound recording and reproduction and, more specif- 
ically, to a resolution-independent system and method 
for making a reducing of a sound and later employing 
the record to synthesize the sound. 

10 

Background Of The Invention 

[0002] Current recording of musical instrument per- 
formances are either based on sampling the analog sig- 
nal of the instrument or recording the gestures that are 15 
input to a controller. This leads to playback situations 
where the performance may only be edited in a fixed 
time domain for the case of sampling, or a performance 
recording that may only realistically record keyboard 
and percussion in the case of Musical Instrument Digital 20 
Interface (MIDI). Additive and spectral synthesis tech- 
nologies break down musical performances into dis- 
crete notes as opposed to a continuous performance. 
Generally, the ability to change the tempo or the key and 
to synchronize a performance with external events dur- 25 
ing playback are often difficult to accomplish at reason- 
able cost and without unacceptable distortions. 
[0003] The sound waveforms produced may be char- 
acterized by many parameters, including frequency and 
amplitude. Using Fourier analysis, sound waveforms 30 
may be represented in a frequency domain as a spectral 
frame, consisting of spectral components. The spectral 
frame contains the waveform's lowest, or fundamental, 
frequency, along with its harmonics (spectral compo- 
nents which occur at multiples of the fundamental fre- 35 
quency). Spectral components from string instruments 
and from vowels in speech typically occur at close to 
whole number multiples of the fundamental frequency, 
while spectral components from percussion instruments 
often occur at non-integral multiples of the fundamental 40 
frequency. 

[0004] Current sound recordings have been seen to 
be typically sample rate dependent or suffer from other 
recording and playback characteristics that make mod- 
ifications to the record difficult to accomplish at accept- 45 
able costs or distortion levels. Additionally, these limita- 
tions make the offering of current sound recordings very 
limited in format for playback selection. Radio stations 
offer a selection of recordings that may be programmed 
for many days into the future with only an occasional so 
specific request capability allowed, usually by tele- 
phone. Even in the case of selected requests, the re- 
cording is completely fixed in format with respect to. 
tempo and key as well as its basic arrangement. 
[0005] Therefore, what is needed in the art is a way 55 
to provide high quality, sample-rate independent record- 
ings that may be selected in a random and expedient 
manner having the capability for specific listener modi- 



fication or adaptation. 
Summary Of The Invention 

[0006] To address the above-discussed deficiencies 
of the prior art, the present invention provides systems 
and methods for recording and synthesizing sound in a 
resolution-independent manner and infrastructures for 
distributing resolution-independent recordings for re- 
mote playback. In one embodiment, one system in- 
cludes: (1 ) a frame generator that extracts fundamental 
frequencies and spectral envelopes from the sound and 
creates frames therefrom and (2) a frame analyzer that 
identifies a selected one of common spectra structures 
and common formant structures in the frames and cre- 
ates a record containing the fundamental frequencies 
and the selected one. 

[0007] The present invention therefore introduces the 
broad concept of storing fundamental frequencies and 
selected structures in sound and creating a record con- 
taining those fundamental frequencies and selected 
structures to provide a basis for subsequent synthesis 
(tantamount to playback). The present invention prefer- 
ably analyzes the sound as a continuous performance 
(irrespective of individual tones or notes). 
[0008] In one embodiment of the present invention, 
the frames are discrete, allowing them to correspond to 
a discrete period of time. In an embodiment to be illus- 
trated and described, common spectra or formant struc- 
tures may be contained in a dictionary to compress the 
total size of the record. 

[0009] In one embodiment of the present invention, a 
musical instrument generates the sound. Those skilled 
in the art are familiar with the formant content of certain 
musical instruments, such as string and wind instru- 
ments. Human voices likewise contain formants that 
may be captured and employed in later synthesis. The 
present invention can operate with any sound, however. 
[0010] In one embodiment of the present invention, 
the frame generator samples the sound before extract- 
ing the fundamental frequencies therefrom. In the em- 
bodiment to be illustrated and described, sampling may 
occur at 1 ms intervals. However, those skilled in the art 
will understand that the present invention is not limited 
to a particular sampling frequency. 
[0011] In one embodiment of the present invention, 
the system further includes a mapping circuit that ap- 
plies a temporal quantization map to the record. Once 
the record is created, the present invention accommo- 
dates a wide range of conventional and later-developed 
sound manipulation techniques. 
[0012] In one embodiment of the present invention, 
the system further includes an editor that modifies a se- 
lected one of a content and an order of the record. The 
editor allows still further manipulation of the sound once 
recorded. 

[0013] In one embodiment of the present invention, 
the frame analyzer identifies the selected one of com- 
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mon spectra structures and common formant structures 
by Fourier analyzing the frames. Those skilled in the art 
are familiar with, in particular, fast Fourier transform 
techniques by which frequencies may be analyzed. The 
present invention is compatible with other conventional 
or later-developed spectrum analysis techniques, such 
as wavelets. 

[0014] The present invention further provides infra- 
structures for distributing recordings for remote play- 
back. One infrastructure includes: (1) a radio station 
having a recording database associated therewith, (2) 
a plurality of recordings contained within the recording 
database, each of the plurality of recordings including a 
selected one of common spectra structures and com- 
mon formant structures corresponding thereto, (3) a re- 
quest receiver, coupled to the recording database, that 
receives remote requests for ones of the plurality of re- 
cordings and (4) a transmitter, coupled to the recording 
database, that transmits the ones of the plurality of re- 
cordings in response to the requests. 
[001 5] The present invention therefore provides what 
amounts to "audio-on-demand" wherein formatted au- 
dio files are provided to remote "radios" to allow the re- 
mote "radios" to synthesize the audio in situ. Therefore, 
in one embodiment of the present invention, the infra- 
structure further includes a plurality of remote radios ca- 
pable of receiving and digitally manipulating the ones of 
the plurality of recordings. The remote radios may com- 
prise software that can be downloaded and executed on 
data processing and storage hardware to allow the ones 
of the plurality of recordings to be played. This infra- 
structure sharply contrasts with conventional analog AM 
or FM radio infrastructures in which remote radios sim- 
ply demodulate and amplify received radio waves. 
[0016] In one embodiment of the present invention, 
the transmitter broadcasts the ones of the plurality of 
recordings to receivers. Alternatively, the ones of the 
plurality of recordings may be addressed to individual 
remote "radios." 

[0017] In one embodiment of the present invention, 
the ones of the pluralities of recordings are embodied in 
a plurality of bitstream files. The bitstream files contain 
data pertaining to the fundamental frequencies and the 
selected one as described above. 
[0018] In one embodiment of the present invention, 
the recording database contains a record of the re- 
quests. This allows song popularity or advertisement 
dissemination to be tracked and accurate royalty pay- 
ments to be calculated automatically. 
[0019] The foregoing has outlined, rather broadly, 
preferred and alternative features of the present inven- 
tion so that those skilled in the art may better understand 
the detailed description of the invention that follows. Ad- 
ditional features of the invention will be described here- 
inafter that form the subject of the claims of the inven- 
tion. 



Brief Description Of The Drawings 

[0020] For a more complete understanding of the 
present invention, reference is now made to the follow- 
5 ing descriptions taken in conjunction with the accompa- 
nying drawings, in which: 

FIGURE 1 illustrates a block diagram of a resolu- 
tion-independent system for recording a musical in- 
fo strument constructed according to the principles of 
the present invention; 

FIGURE 2 illustrates a flow diagram of a resolution- 
independent method of recording a musical instru- 
15 ment that may be carried out in the system of FIG- 
URE 1; 

FIGURE 3 illustrates a block diagram of a resolu- 
tion-independent system for synthesizing a record- 
20 ed musical instrument constructed according to the 
principles of the present invention; 

FIGURE 4 illustrates a flow diagram of a resolution- 
independent method of synthesizing a recorded 
25 musical instrument that may be carried out in the 
system of FIGURE 3; and 

FIGURE 5 illustrates a block diagram of a commu- 
nications infrastructure capable of distributing res- 
30 olution-independent recordings for remote play- 
back. 

Detailed Description 

35 [0021] Referring initially to FIGURE 1, illustrated is a 
block diagram 100 of a resolution-independent system 
for recording a musical instrument constructed accord- 
ing to the principles of the present invention. The reso- 
lution-independent system of block diagram 100 in- 

40 eludes a sound source 110, a sampler 120, a frame gen- 
erator 1 30, a frame analyzer 1 40, a first storage unit 1 50, 
a mapping circuit 160, an editor 170 and a second stor- 
age unit 180. 

[0022] The present invention provides systems and 
45 methods for recording and synthesizing sound in a res- 
olution-independent manner and infrastructures for dis- 
tributing resolution-independent recordings for remote 
playback. In the present embodiment, a musical instru- 
ment may be used to generate the sound. Those skilled 
so in the art are familiar with the formant content of certain 
musical instruments, such as string and wind instru- 
ments. Human voices also contain formants that may 
be captured and employed in later synthesis to be used 
in play-back of the recording. The present invention can 
55 operate with any sound, however. 

[0023] The sampling, which may occur at 1 ms inter- 
vals, is accomplished by the sampler 1 20 before the fun- 
damental frequencies are extracted by the frame gen- 
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erator 130. However, those skilled in the art will under- 
stand that the present invention is not limited to a par- 
ticular sampling frequency or the use of a separate sam- 
pler as shown. The sampler 1 20 may be included as part 
of the frame generator 1 30. This embodiment allows the 5 
data comprising a sound source to be independent of 
the sampling rate or tempo desired. The frame genera- 
tor 130 creates frames and extracts fundamental fre- 
quencies and spectral envelopes from the sound source 
110 through the sampler 120. Then, the frame analyzer 10 
1 30 identifies a selected one of common spectral struc- 
tures and common formant structures in the frames and 
creates a record, containing this selected one and the 
appropriate fundamental frequencies, that is then stored 
in the first storage unit 150. 15 
[0024] The present invention therefore introduces the 
broad concept of storing fundamental frequencies and 
selected structures in sound and creating a record con- 
taining those fundamental frequencies and selected 
structures to provide a basis for subsequent synthesis 20 
to be used in playback. The present invention analyzes 
the sound as a continuous performance, irrespective of 
individual tones or notes. The frames may be discrete, 
allowing them to correspond to a discrete period of time. 
The frame analyzer 140 identifies the selected one of 25 
common spectral structures and common formant struc- 
tures by Fourier analyzing the frames. Those skilled in 
the art are familiar with Fast Fourier Transform (FFT) 
techniques as one technique by which frequencies may 
be analyzed. The present invention is compatible with 30 
other conventional or later-developed spectrum analy- 
sis techniques, such as wavelets, as well. 
[0025] In this embodiment, common spectral or form- 
ant structures may be contained in a dictionary in order 
to compress the total size of the record. The identifica- 35 
tion and grouping of common spectral or formant struc- 
tures allows them to be organized into a record structure 
that may be accessed in a way that is similar to words 
in a special dictionary to be used to reconstruct the par- 
ticular sound composition. The dictionary may typically 40 
be a custom collection of common structures associated 
with the sound source being sampled, framed and ana- 
lyzed. However, the dictionary may also contain com- 
mon structures of a broader collection of sound sources 
that are recognized and tagged to correspond to the par- 45 
ticular sound source being addressed. Further, there 
may be a collection of such dictionaries containing ap- 
propriate spectral and formant structures that are rec- 
ognized and tagged. 

[0026] The resolution-independent system of block so 
diagram 100 further includes the mapping circuit 160 
that applies a temporal quantization map to the record. 
Once the record is created, the present invention ac- 
commodates a wide range of conventional and later-de- 
veloped sound manipulation techniques. The temporal 55 
quantization map allows the record to be synthesized 
with the ability to change tempo or key thereby providing 
different "feel factors". Additionally, it also provides the 



ability to synchronize the playback performance with an 
external clock when other factors dictate. 
[0027] The resolution-independent system of block 
diagram 100 still further includes the editor 170 that al- 
lows still further manipulation of the sound once record- 
ed. The editor 170 may modify the content or the order 
of the record. The spectral structures or the formant 
structures may be modified, if desired, to provide effects 
not contained in the original sound source. The editor 
170 may also be used to re-arrange the record se- 
quence in time relative to the frequency contour. The 
resolution-independent system then stores these 
mapped or edited alternate records in the second stor- 
age unit 180. 

[0028] Turning now to FIGURE 2, illustrated is a flow 
diagram 200 of a resolution-independent method of re- 
cording a musical instrument that may be carried out in 
the system of FIGURE 1. The flowchart 200 illustrates 
a method of recording sound which comprises extract- 
ing fundamental frequencies and spectral envelopes 
from the sound and creating frames from the fundamen- 
tal frequencies and spectral envelopes. Then, one of 
common spectral structures and common formant struc- 
tures in the frames are identified and selected to create 
a record containing both the fundamental frequencies 
and the ones selected. 

[0029] The method begins in a start step 205 wherein 
the decision to create a record is made, and a sound 
source is selected in a step 210 that includes a musical 
instrument which generates the sound. The sound is 
sampled in a step 215 before extracting the fundamental 
frequencies from the sampled sound signal. Frames, 
that may be discrete, are then generated in a step 220 
from which fundamental frequencies and spectral enve- 
lopes are then extracted. These frames are then ana- 
lyzed in a step 225 using Fourier analysis, and then 
common spectral and formant structures are identified 
in a step 230. These common structures are then stored 
in a step 235 which creates a record of the common 
structures. A temporal quantization map may then be 
applied to the record as shown in a step 240, or the 
record may be modified selecting one of a content and 
an order of the record in a step 245 in order to edit the 
contents as required. The method ends in an end step 
250 where the sound has been selected, sampled, 
framed, analyzed, recorded, mapped or modified in this 
embodiment. 

[0030] Turning now to FIGURE 3, illustrated is a block 
diagram 300 of a resolution-independent system for 
synthesizing a recorded musical instrument constructed 
according to the principles of the present invention. The 
resolution-independent system of block diagram 300 in- 
cludes a storage unit 305, a mapping circuit 31 0, an ed- 
itor 315, a waveshaper 320, an output device 325 and 
a speaker 330. The storage unit 305 contains the 
records and dictionaries that have been created by sam- 
pling, framing and analyzing the sound source as dis- 
cussed in FIGURE 1 and FIGURE 2 above. 
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[0031] The mapping circuit 310 applies a temporal 
quantization map to the record. As stated earlier, the 
present invention may accommodate a wide range of 
conventional and later-developed sound manipulation 
techniques allowing the record to be synthesized with a 5 
changed tempo or key and provide the ability to syn- 
chronize the playback performance with an external 
clock. Further manipulation of the recorded sound is 
provided with the editor 315 which may modify the con- 
tent or the order of the record to provide effects not con- 10 
tained in the original sound source. The editor 31 5 may 
also be used to re-arrange the record sequence in time 
relative to the frequency contour as discussed. 
[0032] The waveshaper 320 is coupled to the storage 
unit 305, the mapping circuit 310 and the editor 31 5. The 15 
waveshaper 320 takes the fundamental frequencies 
and applies a waveshaping transfer function to create a 
waveform from either the stored record, the mapped 
record or the edited record. The waveshaper 320 may 
also use some combination of these three in order to 20 
generate the waveform. The waveshaper 320 may also 
select from a number of waveshaping transfer functions 
that are stored in the waveshaper 320 to accommodate 
the waveshaping process. The waveshaper 320 is 
clocked externally, in this embodiment, allowing the 25 
waveform to be synchronized with external events. The 
waveform may then be converted into an output sound 
using the output device 325 and the speaker 330. The 
synthesis process represented here allows the originally 
recorded sound to be reproduced with appropriate fidel- 30 
ity or allows the originally recorded sound to be modified 
as deemed appropriate to the user. This does not pre- 
clude the use of other synthesis techniques such as FFT 
or direct sine. 

[0033] Turning now to FIGURE 4, illustrated is a flow 35 
diagram 400 of a resolution-independent method of syn- 
thesizing a recorded musical instrument that may be 
carried out in the system of FIGURE 3. The method de- 
picted in the flowchart 400 allows a temporal quantiza- 
tion map to be applied or a modification of the content 40 
or the order of a selected record to be accomplished in 
the creation and playback of sound recorded according 
to the present invention. 

[0034] The method begins in a start step 405 wherein 
the decision to synthesize a record is made, and the 45 
record is selected in a step 410. The record is then proc- 
essed using a temporal quantization map in a step 415. 
Then the record is modified through an edit function in 
a step 420. The selected, mapped and edited record is 
then waveshaped in a step 425 and the waveshaped so 
record is then delivered for playback in a step 430. The 
method ends in an end step 435. 
[0035] Turning now to FIGURE 5, illustrated is a block 
diagram 500 of a communications infrastructure capa- 
ble of distributing resolution-independent recordings for 55 
remote playback. The block diagram 500 includes a 
wireless server 505 and an interactive music player 510 
shown in FIGURE 5A, along with a random access play- 



list 515, a download capability 520, a player 525 and a 
speaker 530 for sound reproduction shown in FIGURE 
5B. 

[0036] The present invention further provides infra- 
structures for distributing recordings for remote play- 
back. One infrastructure includes the wireless server 
505 depicted as a radio station having a recording da- 
tabase associated therewith, and a plurality of record- 
ings contained within the recording database, where 
each of the plurality of recordings includes a selected 
one of common spectral structures and common form- 
ant structures corresponding to the records of each in- 
dividual instrument or vocalist in a recording, which is 
re-synthesized and combined in real time during play 
back. 

[0037] The interactive music player 51 0 generates re- 
quests to a corresponding request receiver that is cou- 
pled to the recording database, associated with the wire- 
less server 505, which receives remote requests for var- 
ious ones of the plurality of recordings. Additionally, a 
transmitter, coupled to the recording database, also as- 
sociated with the wireless server 505, transmits the plu- 
rality of recordings in response to the requests. There 
is an additional monodirectional mode in which the re- 
ceiver waits until the desired selection is transmitted be- 
fore downloading and playing it. 
[0038] The present invention therefore provides what 
amounts to audio-on-demand wherein formatted audio 
files are provided to remote radios or players, which act 
as "client" receivers, to allow these remote radios to syn- 
thesize the audio in situ. In this embodiment of the 
present invention, the infrastructure further includes a 
plurality of remote digital radios capable of receiving and 
digitally manipulating the plurality of recordings. This in- 
frastructure sharply contrasts with conventional analog 
AM or FM radio infrastructures in which remote radios 
simply demodulate and amplify received radio waves. 
This infrastructure may function with any currently-pro- 
posed or later-developed digital transmitter and receiver 
standards. The prograr material for the recordings may 
include but is not limited to weather, news, stock quotes 
or other topical information. 

[0039] The random access playlist 515 represents the 
pluralities of recordings, which are embodied in a plu- 
rality of bitstream files. The bitstream files contain data 
pertaining to the fundamental frequencies and the se- 
lected one as described above. The bitstream files, 
which may represent a collection of selections or the col- 
lection of offerings, may occur in a single serial loop. 
The user may select the ones of these that are down- 
loaded and played. Alternately, they may occur in a col- 
lection of parallel loops allowing the user to perform the 
download 520 more rapidly. The transmitter, associated 
with the wireless server 505, may broadcast the ones of 
the plurality of recordings to all remote radios. Alterna- 
tively, the ones of the plurality of recordings may be ad- 
dressed only to individual remote radios. The recording 
database may contain a record of the requests. This al- 
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lows song popularity or advertisement dissemination to 
be tracked and accurate royalty payments to be calcu- 
lated automatically. 



Claims 

1. A method of recording sound, for example sound 
produced by a musical instrument, comprising: 

extracting fundamental frequencies and spec- 
tral envelopes from said sound; 
creating frames from said fundamental fre- 
quencies and spectral envelopes; 
identifying a selected one of common spectra 
structures and common formant structures in 
said frames; and 

creating a record containing said fundamental 
frequencies and said selected one. 

2. The method as recited in claim 1 wherein said 
frames are discrete. 

3. The method as recited in claim 1 or claim 2 further 
comprising sampling said sound before extracting 
said fundamental frequencies therefrom. 

4. The method as recited in any of the preceding 
claims further comprising applying a temporal 
quantization map to said record. 

5. The method as recited in any of the preceding 
claims further comprising modifying a selected one 
of a content and an order of said record. 

6. The method as recited in claim 8 wherein said iden- 
tifying step includes Fourier analyzing said frames. 

7. A system for recording sound, for example sound 
produced by a musical instrument, comprising 
means arranged to carry out a method as claimed 
in any of the preceding claims. 

8. An infrastructure for distributing recordings for ex- 
ample recordings of sounds produced by musical 
instruments, for remote playback, comprising: 

a radio station having a recording database as- 
sociated therewith; 

a plurality of recordings contained within said 
recording database, each of said plurality of re- 
cordings including fundamental frequencies 
and a selected one of common spectra struc- 
tures and common formant structures corre- 
sponding thereto; 

a request receiver, coupled to said recording 
database, that receives remote requests for 
ones of said plurality of recordings; and 



a transmitter, coupled to said recording data- 
base, that transmits said ones of said plurality 
of recordings in response to said requests. 

5 9. The infrastructure as recited in claim 8 wherein said 
transmitter broadcasts said ones of said plurality of 
recordings to receivers. 

10. The infrastructure as recited in claim 15 further 
io comprising a plurality of remote radios capable of 

receiving and digitally manipulating said ones of 
said plurality of recordings. 

11. The infrastructure as recited in any of claims 8 to 
15 10 wherein said ones of said pluralities of record- 
ings are embodied in a plurality of bitstream files. 

12. The infrastructure as recited in any of claims 8 to 11 
wherein said recording database contains a record 

20 of said requests. 

13. A radio, comprising: 

a receiver for receiving a recording including 
25 fundamental frequencies and a selected one of 

common spectra structures and common form- 
ant structures corresponding thereto; 
a waveshaper, coupled to said receiver, for ap- 
plying a waveshaping transfer function based 
30 on said selected one to said fundamental fre- 

quencies to create a waveform; and 
a speaker, coupled to said waveshaper, for con- 
verting said waveform into an output sound. 

35 14. The radio as recited in claim 13 further comprising 
a mapping circuit, coupled to said receiver, that ap- 
plies a temporal quantization map to said selected 
one. 

40 15. The radio as recited in claim 13 or claim 14 wherein 
said waveshaping transfer function is selected from 
a plurality of waveshaping transfer functions stored 
in said waveshaper.. 

45 16. The radio as recited in any of claims 13 to 15 where- 
in said waveshaper is clocked externally. 

1 7. The radio as recited in any of claims 1 3 to 1 6 further 
comprising an editor, coupled to said receiver, for 
50 modifying a content of said recording. 
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